Skip to content

feat(partitions): implement partition background tasks#3564

Merged
numinnex merged 6 commits into
masterfrom
partition_background_workers
Jun 30, 2026
Merged

feat(partitions): implement partition background tasks#3564
numinnex merged 6 commits into
masterfrom
partition_background_workers

Conversation

@numinnex

@numinnex numinnex commented Jun 25, 2026

Copy link
Copy Markdown
Contributor

This PR adds support for background tasks for partitions (segment cleaners). Additionally it adds support for DeleteSegment and PurgeTopic.

@github-actions github-actions Bot added the S-waiting-on-review PR is waiting on a reviewer label Jun 25, 2026
@codecov

codecov Bot commented Jun 25, 2026 •
edited
Loading

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 33.92405% with 522 lines in your changes missing coverage. Please review.
✅ Project coverage is 74.06%. Comparing base (688d32a) to head (38a24b5).

Files with missing lines Patch % Lines
core/partitions/src/iggy_partition.rs 36.76% 215 Missing ⚠️
core/metadata/src/stm/stream.rs 52.41% 64 Missing and 5 partials ⚠️
core/server-ng/src/dispatch.rs 0.00% 69 Missing ⚠️
core/metadata/src/impls/metadata.rs 0.00% 41 Missing and 1 partial ⚠️
core/shard/src/router.rs 0.00% 34 Missing ⚠️
core/shard/src/lib.rs 0.00% 32 Missing ⚠️
core/server-ng/src/segment_cleaner.rs 0.00% 29 Missing ⚠️
core/server-ng/src/bootstrap.rs 0.00% 16 Missing ⚠️
core/partitions/src/iggy_partitions.rs 0.00% 9 Missing ⚠️
core/server-ng/src/partition_reconciler.rs 80.00% 2 Missing and 4 partials ⚠️
... and 1 more

❌ Your patch check has failed because the patch coverage (33.92%) is below the target coverage (50.00%). You can increase the patch coverage or adjust the target coverage.

Additional details and impacted files 
@@             Coverage Diff              @@
##             master    #3564      +/-   ##
============================================
- Coverage     74.35%   74.06%   -0.30%     
  Complexity      937      937              
============================================
  Files          1257     1258       +1     
  Lines        130671   131452     +781     
  Branches     106537   107363     +826     
============================================
+ Hits          97156    97354     +198     
- Misses        30434    30952     +518     
- Partials       3081     3146      +65
Components Coverage Δ
Rust Core 74.73% <33.92%> (-0.33%) ⬇️
Java SDK 62.44% <ø> (ø)
C# SDK 71.40% <ø> (-0.71%) ⬇️
Python SDK 88.88% <ø> (ø)
PHP SDK 84.29% <ø> (ø)
Node SDK 91.13% <ø> (+0.01%) ⬆️
Go SDK 40.14% <ø> (ø)
Files with missing lines Coverage Δ
core/binary_protocol/src/consensus/operation.rs 96.55% <100.00%> (+0.12%) ⬆️
core/binary_protocol/src/dispatch.rs 91.63% <ø> (ø)
core/metadata/src/stm/snapshot.rs 86.01% <100.00%> (+0.11%) ⬆️
core/partitions/src/segment.rs 93.54% <100.00%> (+38.37%) ⬆️
core/consensus/src/observability.rs 41.00% <0.00%> (-0.10%) ⬇️
core/server-ng/src/partition_reconciler.rs 86.69% <80.00%> (-0.21%) ⬇️
core/partitions/src/iggy_partitions.rs 68.43% <0.00%> (-4.92%) ⬇️
core/server-ng/src/bootstrap.rs 10.76% <0.00%> (-0.10%) ⬇️
core/server-ng/src/segment_cleaner.rs 0.00% <0.00%> (ø)
core/shard/src/lib.rs 72.43% <0.00%> (-2.41%) ⬇️
... and 5 more

... and 36 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

atharvalade
atharvalade previously requested changes Jun 25, 2026
View reviewed changes
Comment thread core/partitions/src/iggy_partition.rs
Comment thread core/partitions/src/iggy_partition.rs Outdated
Comment thread core/metadata/src/stm/stream.rs
@github-actions github-actions Bot added S-waiting-on-author PR is waiting on author response and removed S-waiting-on-review PR is waiting on a reviewer labels Jun 25, 2026
@numinnex

numinnex commented Jun 29, 2026

Copy link
Copy Markdown
Contributor Author

/ready

@github-actions github-actions Bot added S-waiting-on-review PR is waiting on a reviewer and removed S-waiting-on-author PR is waiting on author response labels Jun 29, 2026
@numinnex

numinnex commented Jun 30, 2026

Copy link
Copy Markdown
Contributor Author

/ready

@numinnex

numinnex commented Jun 30, 2026

Copy link
Copy Markdown
Contributor Author

@atharvalade I've addressed all of your comments, if you don't have more comments lets procede with merging this PR.

@numinnex numinnex dismissed atharvalade’s stale review June 30, 2026 14:12

All of the comments were resolved.

@numinnex numinnex merged commit 14f0ea2 into master Jun 30, 2026
160 of 164 checks passed
@numinnex numinnex deleted the partition_background_workers branch June 30, 2026 14:13
@github-actions github-actions Bot removed the S-waiting-on-review PR is waiting on a reviewer label Jun 30, 2026
mfyuce pushed a commit to mfyuce/iggy that referenced this pull request Jul 1, 2026
@numinnex @mfyuce
feat(partitions): implement partition background tasks (apache#3564)
daa7421 
This PR adds support for background tasks for partitions (segment
cleaners). Additionally it adds support for `DeleteSegment` and
`PurgeTopic`.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@spetz spetz spetz approved these changes
@mmodzelewski mmodzelewski mmodzelewski approved these changes
@atharvalade atharvalade atharvalade left review comments

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@numinnex @spetz @mmodzelewski @atharvalade